1. Field
The present invention relates generally to processing of multimedia data signals and, more specifically, to interpolating frames in a video data stream.
2. Description
Digital video data streams can be processed by various types of electronic components, such as computers, digital televisions, digital video cameras, video conferencing equipment, etc. In order to ease the requirements for transmitting digital video from one component to another, the data is typically encoded and compressed according to one of several well known methods at the sending end of a communications path. This functionality is usually provided by an encoder within a first component. At the receiving end of the communications path, the encoded and compressed video stream is then typically decompressed and decoded. This functionality is usually provided by a decoder within a second component. The encoder and decoder for any given component are often combined into one unit as a video encoder/decoder, or video codec.
Various methods are known in the art for improving the quality of digital video as handled by video codecs. Generally, the quality of a video stream may be improved by increasing the resolution of each frame of a video stream, increasing the frame size, or by increasing the frame rate of the video stream. In one known method, called frame rate upsampling (FRU), fewer frames per second are transmitted from a sender to a receiver, but the receiver analyzes the received frames of the stream and generates new intermediate frames based on the analysis. When transmission happens in a fixed channel, fewer frames per second means that more bits will become available to encode each individual frame. This translates into fewer but sharper frames. The video quality is therefore improved due to the increased frame rate without the need for additional bandwidth for transmission of the added frames. In some cases, the transmitted frames may be sharper than before, so the overall transmission bandwidth may be approximately the same, but with FRU being performed by the receiver a larger image may be rendered.
Although FRU methods have been and continue to be developed, existing solutions exhibit a number of problems affecting image quality. Digital video streams rendered by a receiver after FRU processing often have noticeable block artifacts which detract from the image quality. For some video content, such as sports and music videos having a substantial amount of fast motion, FRU produces unsatisfactory results. In some instances, the newly interpolated frames are deemed to be visually unacceptable for fitting in to the video stream and must be discarded prior to rendering. In addition, the frame size and frame rates at which current FRU techniques are effective is limited. Therefore, there is a need in the digital video art for techniques that overcome these and other disadvantages.
An embodiment of the present invention is a method of interpolating a new frame between a previous frame and a current frame of a video stream. The method includes identifying nodes and edges of triangles present in the previous frame, constructing a superimposed triangular mesh of the previous frame from the identified nodes and edges, estimating displacement of nodes in the superimposed triangular mesh from the previous frame to the current frame, and rendering the new frame based on the estimated displacement of nodes.
Another embodiment of the present invention is an apparatus for frame rate upsampling based on a previous frame and a current frame of a video stream. The apparatus includes a node and edge identification component to identify nodes and edges of triangles present in the previous frame, and a triangulation component to construct a superimposed triangular mesh of the previous frame from the identified nodes and edges. The apparatus further includes a node displacement estimation component to estimate displacement of nodes in the superimposed triangular mesh from the previous frame to the current frame, and a renderer to render the new frame based on the estimated displacement of nodes.